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Abstract 

The use of open-loop coding can be easily extended to a closed-loop concatenated code if the channel has access 
to feedback. This can be done by introducing a feedback transmission scheme as an inner code. In this paper, this 
process is investigated for the case when a linear feedback scheme is implemented as an inner code and, in particular, 
over an additive white Gaussian noise (AWGN) channel with noisy feedback. To begin, we look to derive an optimal 
linear feedback scheme by optimizing over the received signal-to-noise ratio. From this optimization, an asymptotically 
optimal linear feedback scheme is produced and compared to other well-known schemes. Then, the linear feedback 
scheme is implemented as an inner code to a concatenated code over the AWGN channel with noisy feedback. This 
code shows improvements not only in error exponent bounds, but also in bit-error-rate and frame-error-rate. It is 
also shown that the if the concatenated code has total blocklength L and the inner code has blocklength, A'^, the 
inner code blocklength should scale as A'^ = O (^), where C is the capacity of the channel and R is the rate of 
the concatenated code. Simulations with low density parity check (LDPC) and turbo codes are provided to display 
practical applications and their error rate benefits. 

Index Terms 

concatenate coding, Schalkwijk-Kailath coding scheme, additive Gaussian noise channels, linear feedback 

L Introduction 

The field of open-loop error-correction coding has been rich with innovations over the last 10-20 years with 
implementation of codes like turbo codes and low density parity check (LDPC) codes. These codes have proven 
that open-loop methods can be very powerful. However, an important question to be asked is: "Can we do better 
with closed-loop coding?" In this paper, we investigate the use of closed-loop concatenated coding (see Fig. [T]| 
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Fig. 1. Closed-loop concatenated coding system. 



over an additive white Gaussian noise (AWGN) channel with noisy feedback. The benefits of this have ah^eady 
been shown for a noiseless feedback channel as in fT). 

By definition, concatenated coding consists of two codes: an inner code and an outer code. As for the outer 
code, we will assume it is any general forward error-correction code as to make this method applicable to any 
open-loop technique. Furthermore, since we are interested in closed-loop coding, the inner code will be a feedback 
transmission scheme; this, however, creates the intermediate goal of designing a good feedback scheme. In this, we 
narrow our focus to the class of linear feedback transmission schemes- meaning the each transmission is a linear 
function of feedback side-information and the message to be sent. With perfect feedback, this class is known to 
have low complexity and high reliability ||2l, ID. Therefore, we will try to exploit these advantages even with a 
noisy feedback channel. 

The search for the best linear feedback coding scheme for AWGN channels has a long history, dating back to 
1956 with a paper by Elias ID. However, most early work was done in the late 1960's with papers like Hl-lTl. In 
1966, SchaUcwijk and Kailath developed a specific linear coding technique that utilizes a noiseless feedback channel 
II2I, O. The encoding scheme was based off of a zero-finding algorithm called the Robbins-Monro procedure which 
sequentially estimates the zero of a function given noisy observations. Because of its low complexity, much work 
has been done extending and evaluating the performance of the Schalkwijk-Kailath (S-K) scheme in different 
circumstances. The performance was examined when there is bounded noise on the feedback channel in JSl. In 
121, ifTOl . the system was observed under a peak energy constraint. A generalization of the coding scheme for 
first-order autoregressive (AR(1)) noise processes on the forward channel was derived in (|5]. The use of the coding 
technique was extended to applications in stochastic controls in ||6l. The scheme was used in ifTTI for a derivation 
of feedback capacity for first-order moving average (MA(1)) channels. In |fT2l, the scheme was rederived using a 
previous result in |i4j and then altered for specific use with PAM signaling. Variations on the scheme were created 
by using stochastic approximation in |[T3l . The S-K scheme was used in a derivation of an error exponent for 
AWGN channels with partial feedback in lfT4l . Due to such popularity, we will use this scheme for performance 
comparisons. 

Recently, the area of general feedback communication schemes has been also studied as in ifTsl . 1161 . These use 
a technique called Posterior Matching in which information at the receiver is refined using the a-posteriori density 
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function which is matched to the input distribution function. Such techniques have also proven to be capacity- 
achieving and, in fact, a generalization of the S-K scheme. However, these schemes along with the S-K scheme 
aU rely on the presence of a noiseless feedback channel to achieve non-zero rate. If noise is present, all of these 
schemes have only an achievable rate of zero. 
In this paper, we do the following: 

• Give the basic framework of concatenated coding and design a linear feedback scheme for use as an inner 
code by: 

- Using a matrix formulation for feedback encoding, we formulate the maximum SNR optimization problem. 
The formulation consists of a combining vector and noise encoding matrix. It shares many similarities to 
the method employed by [5 |. In addition, an upper bound on SNR for all noisy feedback schemes over 
the AWGN channel is derived and shown to be tighter than the bound previously made by lH]. 

- Using SNR as the cost function of interest, we solve for i) the SNR-maximizing linear receiver given a 
fixed linear transmit encoding scheme and ii) the SNR-maximizing linear transmitter given a fixed linear 
receiver. 

- Using insights from the numerical optimization, we derive what we believe to be the optimal linear 
processing set-up. This performance of the proposed scheme approaches the linear processing SNR upper 
bound as the blocklength grows large. 

• Using the proposed linear feedback scheme, we then implement a concatenated code over the AWGN channel 
with a general error-correction code as an outer code. The error exponent for the concatenated scheme is 
derived in terms of the error exponent for the outer code. 

• Upper and lower bounds on the feedback error exponent are then derived using this setup. These bounds are 
then used to illustrate the effect of using the proposed hnear feedback scheme as an inner code. An approximate 
trade-off between inner code blocklength and total code blocklength is also derived. 

• Simulations are run to show advantages in bit-error-rate (BER) and frame-error-rate (PER) when the outer 
code is either a turbo code or LDPC code. 

The paper is organized in the following manner. The overall system and the framework for a general closed-loop 
concatenated coding scheme are introduced in Section II. In Section III, the concept of linear feedback coding is 
introduced to develop an appropriate inner code for the overall concatenated scheme. Two methods of optimization 
for a general linear coding scheme are also briefly introduced. Using these optimization methods, a linear feedback 
scheme is proposed in Section IV. This section also consists of analyzing the asymptotic performance of our 
scheme, along with deriving alternate proofs of results from related papers. In Section V, the proposed linear 
feedback scheme is compared against the S-K scheme to illustrate gains in performance. Section VI introduces the 
concatenated coding scheme and its error rate analyses. Simulations are then given in Section VII to demonstrate 
practical concatenated code performance with turbo codes and LDPC codes. 
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II. General Closed-Loop Concatenated Coding 

Looking at the AWGN channel, at channel use k — 1,2, ... ,L, the transmitted signal is x[k] and the receiver 
obtains 

y[k]=x[k]+z[k]. (1) 

For our purposes, we will assume are i.i.d. such that z[k] ^ J\f{0,l). Also, to remain practical, we can 

impose an average transmit power constraint, p such that 

^[x^x] < Lp, (2) 

where x = x[2], . . . , a;[i]]"^ . 

Consider sending a length K open-loop code across the AWGN channel with noisy feedback. The transmission 
of each component of the codeword, c = [ci,C2, . ■ . , ck], will be encoded using an inner code of blocklength N 
that has access to noisy feedback. Thus, the total concatenated codeword and accordingly, the transmit vector, x, 
has length L = KN. If we write the components of the inner code as Si{cj) (the i-th inner code component used 
to encode the j-th outer code component), then x can alternatively be written as 

X= [Siici), S2{ci), . . . , Sn{ci), Si{c2), ■ . . , SNicK)] ■ 

This encoding process now can be grouped by the concatenated encoder (or superencoder) in Fig. [1] Using the 
power constraint, (|2]i, it is possible to also bound the average power of the outer codeword as 

E[c^c] < Kp. (3) 

After all transmissions have been made, the inner decoder creates an estimate of the current outer code codeword 
by decoding y, N entries at a time. This process produces the following total codeword estimate 

C=[ci,C2,...,Ck], (4) 

which will be passed on to the outer decoder for final decoding. 

This setup now allows for a very convenient simplification of the concatenated coding scheme. After processing 
at the inner code decoder (which will be described in Section IV), the channel given in ([T]i can be seen alternatively 
as 

=c[fc]+5[fc], (5) 

as seen in Fig. |2] The modified noise component, z[k], has a new variance dependent on the properties of the 
inner code. In fact, the whole effect of the inner code is encapsulated in the modified noise, z[k]. Due to the inner 
code being undefined at this point, we cannot go into more depth. However, a per-component signal-to-ratio can 
be calculated as 



Since the noise is i.i.d., this is the same for all This implies that yar(0[fc]) 



p 



SNRa. 
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Fig. 3. A transmitter/receiver pair over an AWGN cliannel with noisy feedback. 



This technique has converted the closed-loop problem now into an open-loop problem. Note that this simplification 
is the exact same process as defining the inner code and channel together as a superchannel as discussed in |T7| . 
Since the outer code is a general forward error-correction code, this equivalent mapping has greatly reduced the 
problem to now finding the length N inner code that maximizes SNR^ and, thus, minimize the modified noise 
on the channel. In the next few sections, this will be our exact focus. 

III. The Inner Code: Linear Feedback Coding 

As stated above, the focus of this section is to design a length N inner code that maximizes received SNR. 
Since we have the availability of feedback for the inner code (see Fig. [T]i and it is a main focus of the paper, 
we will utilize feedback side-information at the inner code encoder. With this setup, it is possible to employ 
linear feedback encoding - the advantages of which were described in the Section I. To begin, the focus is now 
narrowed to only the inner code encoder/decoder pair; hence, we will only be considered with sending and receiving 
one codeword of the inner code (i.e., [si(ci), S2(ci), . . . , SAr(ci)]). This corresponds to looking at channel uses 
k ~ {{i — 1)N + 1), . . . , {i)N. For simplicity, we study the case when i = 1. To begin, the notion of general linear 
feedback coding is introduced. Because of the focus of this section and to ease with reference, we to refer to the 
inner code encoder as the transmitter and the decoder as a receiver 
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A. General Linear Feedback Encoding 

In this section, we introduce the general framework of a linear feedback coding scheme in a linear algebraic 
formulation (similar to IS)). A feedback channel allows the transmission of data from the receiver back to the 
transmitter. Considering the system in Fig. [3] we see that such a link is available with unit delay and additive noise. 
As in Section II, at channel use k — 1,2, ... ,A^, x[k] is sent from the transmitter across an AWGN channel and 
the receiver receives 

y[k]^x[k] + z[k], (7) 

where {z[fc]} are i.i.d. such that each z[k] ^ Af{0, 1). Because of the feedback channel, the transmitter also has access 
to side-information. In this case, we assume the side-information to be the past values of y[k] corrupted by additive 
noise, n[k]. We assume that {ri[fc]} are i.i.d. such that n[k] ^ A/'(0, rr^) and are independent of . Since 

we are designing an encoding scheme that will utilize feedback, x[k] is encoded at the transmitter using the noisy 
side information {y[l] + n[l],y[2] + n[2], . . . ,y[k — 1] + n[k — 1]}. By removing the known transmitted signal 
contribution, this is equivalent to encoding with side information {z[l] + n[l], z[2] + n[2], . . . , z[k — 1] + n[k — 1]}. 

We now describe a general coding scheme that utilizes this channel and feedback configuration. The goal of the 
coding scheme is to reliably send a component of the outer code codeword Ci from transmitter to receiver across 
an additive noise channel using N channel uses. However, to broaden the applications of the developed scheme, 
the message to be sent will be assumed to be a general 9 E R. We assume the symbol 9 is chosen from the set 
Q ~ {di,92, ■ ■ ■ ,6m} where M is the number of symbols and each symbol is equally-likely. Furthermore, we 
assume that 9 is zero mean and that the second moment of 9, E[9'^], is known. Due to the fact that only received 
SNR and rate of transmission calculations will be performed, the above description of the source alphabet proves 
sufficient. 

With this set-up, the input to the receiver can be written as 

y = x + z, (8) 

where, as above, the notation x refers to x = [a::[l], x[2], . . . , a;[Af]]"^. Because of the total average transmit power 
constraint (|2]), the transmitted power of the signal x (for N transmissions) is bounded by 

£;[x^x] < Np. (9) 

The output of the transmitter x is given as 

X = F(z + n) + g6», (10) 

where g e is a unit vector and F e M^^^ is a matrix called the encoding matrix. F is of the form 

" " 




fw,! ■ ■ ■ fN,N-l 
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which is referred to as strictly lower-triangular to enforce causality. Taking a closer look at ( fTOl ). we see that this is 
exactly the linear processing model - each x[k] is a linear function of past values of {z[k] + n[k]} and the message, 
d. 

Now, consider the processing at the receiver's end. The input to the receiver y is given by (HJ. Using (fTol i. (HJ 
becomes 

y = F(z + n)+ge' + z= (I + F)z + Fn + g6'. (11) 

After all N transmissions have been made, the receiver combines all received values as a linear combination and 
forms an estimate of the original message, 9. This operation is written as 

= q^y, 

where q e is a vector called the combining vector. It is now evident that a general hnear feedback scheme can 
be completely described in terms of F,q, and q. In fact, the S-K scheme can be described in this way, but since 
it's definition is not necessary, it will be pushed to Appendix A. 

As an aside, it is important to note that up until this point, a specific decoding process has not been specified. 
However, since we will be passing on the output straight to the outer decoder, we choose only to perform only soft 
decoding; hence the estimate will be sent straight to the outer decoder without mapping it to an output alphabet. 
Of course, minimum-distance decoding (and similar techniques) can be easily implemented for hard decoding. 

Looking back at the transmitted signal, it proves helpful to study how much power is used sending the message 
and how much is dedicated to encoding noise for noise-cancellation at the receiver. This can be examined by noting 
that the average transmitted power is 

Elx^x] ^ tr{FE[{z + n){z + nf]F'^) + \\gfE[e^] 
= (l + a2)||F||^ + E\e^ 

noise-cancellation power signal power 

< Np, 

where \\F\\l^J2flr 

Because the sum of the noise-cancellation power and signal power must be less than Np, we introduce a new 
variable that will be a measure of the amount of power used for noise-cancellation. To accomplish this, let us 
introduce 7 e K such that < 7 < 1. Using the power allocation factor 7, let E[9'^] be scaled such that 

E[0^] = (1 - j)Np, (12) 

and F be constrained such that 

il + (T^)\\F\\l<Njp. (13) 
Until Section IV, it is now assumed that 7 is fixed. 
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B. Optimization of Received SNR 

As in Section II, our goal is to create a scheme that maximizes the received signal-to-noise ratio. Not surprisingly, 
we have chosen it to be our main performance metric. It can be derived by noting the form of the receiver's estimate 
of the transmitted message. The received signal after combining is 

0^:=qTy = q'J^((I + F)z + g6l + Fn). (14) 

It follows that the received SNR is 

SNR^ 



E[\cfge\'] 



E[\qT(I + F)z + qTFn\y 
^ E[9^] |q^g|^ 

||q^(I + F)||'+a2|lq^F|r 
It would be ideal to optimize the SNR expression over all F, q, and q. However, this method turns out to be 

quite intractable. Instead, we focus on optimization by two conditional optimization techniques that maximize SNR 
either given F or given q. Since the derivations of these methods are not necessary for our discussions, their proofs 
are pushed to Appendix B. Note that the following procedures are hardly groundbreaking, but are given as lemmas 
to aid in later reference. The first lemma is introduced to design F to maximize the received SNR for a given q. 

Lemma 1. Given a combining vector q and the power constraint given in ( 1731 ), the F that maximizes received 
SNR can be constructed using the following procedure: 

1) Define q(j) = [ (j^+i, 9^+2, ■■■,qN ]^ where l<i<N-\, 



2) Construct the entries of F, /j j-, as 



3—-, * > J 



i 0, i<j 
where A € M « the smallest A > such that ||r|j^ < (1 + a'^)^^Njp. 

The next lemma provides the symmetrical result; it constructs a q that maximizes the received SNR given F. 

Lemma 2. Given an encoding matrix, F, the q that maximizes received SNR, Qopt, can be found by letting q be 
the eigenvector vector of (I + F)(I + F)"^ + c^FF"^ that corresponds to its minimum eigenvalue. 

Note that this lemma has well-known analogous results in estimation theory. In brief, the optimal q is created by 
forming the projection 

q = rn-i ^ (16) 
g^C ig 

where C = (I + F)(I + F)^ + cr^FF^. However, it can be shown that to choose g to maximize SNR, then ( fT6l ) 
implies that q = g and both should be chosen as in Lemma 2. In addition, one can show that given F, the definition 
of q in Lemma 2 produces the minimum variance unbiased (MVU) estimator. To illustrate, the variance of the 
estimator, 9 is 

Var{e) = E[{0 - 0f] = [(I + F)(I + F)'^ + cr^FF^] q. 
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Since q has already been chosen to minimize this quantity and it is unbiased {E[9] = 9), it is the MVU estimator 
(consequentially also the least squares estimator). 

These two lemmas will now provide sufficient for developing a linear feedback scheme that maximizes the 
received SNR as in Section IV. 



C. Upper Bound on Rate and Received SNR 

Due to the development of Lemmas 1 and 2, we can now apply them to construct some interesting results on 
the class of linear feedback codes. First, an upper bound on received signal-to-noise ratio is found. 

The method used in Lemma 1 to maximize the received SNR focuses on minimizing the denominator of (flSl l. It 
does so while also compensating for the average power constraint given in (|9]l. If this constraint is relaxed to allow 
the denominator of the SNR to be minimized completely, it is possible to derive an upper bound on the received 
SNR. 

Lemma 3. The received SNR for a linear feedback encoding scheme with feedback noise variance, a^, is bounded 
by 

SNR < i-t^iVp (17) 

Proof: Looking at the proof of Lemma 1 (in Appendix B), the goal is to maximize the received SNR by 
minimizing the denominator in ( fTSl l. However, the average power constraint in ( fT3] ) restricts the optimization 
problem and the solution is not optimal in a least-squares sense. If the power constraint is removed, ( |82] ) becomes 

hopt = argmin || Ab - q||' + || Abf . (18) 

b 

This results in the solution to the least-squares problem being 

b-((l + f72)A^A)-iA^q. 
Using this b to construct F, dSTI ) becomes 

||q^(I + F)f =X:(9.-YT^)^+4 (19) 



Similarly, the other noise term is 



N-l 



i=l 



^ n / 2,2 - (22) 
(1 + a^y 
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Using these two results, the received SNR, using ( fTSl ). can be written as 



SNR < (23) 



( Y I 



1 + 0-' 



2 



E[9^] (24) 



1 + ^2 

< ^Np (25) 



In ||5l, another upper bound is given for linear feedback schemes with noise on the feedback channel. Using the 
notation consistent with the above formulations, the Butman bound can be given by: 



SNR < S[x^x] + i^S[y^y], 



(26) 

= i^iVp + 2||F||2, + iI. (27) 

Since the last two terms in the inequality are strictly greater than zero, this bound is strictly greater than ( |25]) . 
implying a helpful tightness in the new bound in Lemma 3. 

Suppose that we now allow the size of the symbol set, 8 = {di, 02, ■ ■ ■ , Om}, to be a function of the blocklength 
(i.e., Af^^)). The rate in bits per channel use of our linear encoding is defined as = \og2{M^^'>)/N (Note 
that r is used instead of R to emphasize that this only applies to the inner code). A rate r = liniAr^oo r^'^^ is said 
to be achievable if the probability of error goes to zero as N oo. Also, if the linear feedback scheme is viewed 
as a superchannel (as in Fig. |2l), the received SNR for the linear feedback scheme can be seen as the received SNR 
for the superchannel. Thus, the capacity of the superchannel is ^ log2(l + SNR), where SNR is the received SNR 
for the linear feedback scheme in use. Now using the SNR bound result, we can construct an alternate proof of 
Proposition 4 given in ifTsl . 

Lemma 4. Given any linear feedback coding scheme of rate r over an AWGN channel with noisy feedback, if r is 
achievable then r — Q. 

Proof: As above, if regarding the linear feedback scheme over the AWGN channel as a superchannel, the 
capacity is 

C-ilog2(l + 57Vi?), (28) 
where SNR is the received SNR of the linear feedback scheme. Then, any achievable rate r must satisfy 

~ N^oo N 

= 0. (30) 
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IV. A Linear Feedback Coding Scheme 

Now, we use both methods presented in Lemmas 1 and 2 as iterative optimization tools. Using Lemma 1, we 
can design F to maximize the received SNR. We can do the same using Lemma 2 to design q. However, it is 
desirable to optimize q and F jointly to maximize the SNR. Consider being given an initial combining vector, q*^"-' . 
Using Lemma 1, we can design an encoding matrix F*^''' to maximize the received SNR. Now, that F*-*^^ has been 
constructed, we can use Lemma 2 to further maximize the received SNR by designing q*^^'. This process can be 
repeated until the received SNR does not increase with an iteration (i.e., we have reached a fixed point). 

After repeatedly using this algorithm for different q*^*^) and different values of N and p, a pattern emerges. The 
structures of both F and q are the same for every scheme that maximizes the received SNR. Using random search 
techniques, we were unable to find an alternate form that produced a higher received SNR. Thus, empirically, the 
problem appears convex - the same result was produced independent of the random initial vector, q*^*^'. In the 
following conjecture, we propose that these structures of F and q give the scheme that maximizes the received 
SNR. 

A. The Feedback Scheme 

Conjecture 1. Consider again the system from Fig. \3\ Then, given the power constraints in ( 1731 ) and ( Ii2l ). the F 
and q that maximize the received SNR are of the following forms: 

• Y is a strictly lower diagonal matrix with all entries along the diagonals being equal ( also called a Toeplitz 
matrix), 

. {l + a^)\\F\\j,^Njp, 

• For some /3 € M such that f3 G (0, 1), the form of q is 



y 1 - /?2 

Note that the term multiplying the vector q is for normalization purposes. 

Assuming that this form is optimal, we can solve for the optimal /3 and the entries of F. 

Lemma 5. Given the power constraints in ( 1751 ) and ( 1721 ), F and q have the following definitions given the forms 
in Conjecture 1: 

1) The optimal j3, /3o, is the smallest positive root of 

-{N+{l + a^)Njp)/3^ + {N-l), (31) 

2) 
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3) 





"(1+t2)/3o 

1 + 0-2 







■ (l+<T^)/3o 

The proof of Lemma 5 is given in Appendix C. 

Because a closed-form solution of /?o is not readily available, it proves very useful to define a close approximation. 
Solving for (3 in ( |92] i. we get 

rwTW^ (32) 



/3o 



N +{l + a'^)N-fp 



Since /? €E (0, 1) we can assume that j3 << 1 which gives us the approximation (denoted /?i), 



/3o 



iV- 1 



(33) 



iV+(l + (T2)7V7p Yl + (l + cr2)7p 
The approximation, /3i, can be derived alternatively using iterative fixed point techniques. This method also produces 
a bound on the deviation from /3o. However, for values of iV > 5, this approximation becomes extremely close. 

It can be shown using (fTsl l, that the received SNR for this scheme (now explicitly notating that the SNR is a 
function of /3 and 7) is 



57Vi?(/3,7) = 



{l + a')N{l-^)p 



(34) 



cr2 ^ p2{N-l) 

It is important to note that using /3i, the scheme exceeds the power constraint in (|9]) by a small amount that dies 
away as the blocklength gets larger According to our power constraints, ||F||^ < (1 + a'^)^^N^p. However, using 
/?! to build the scheme we get 



(1+^2)2 



+ [l+(T^)-^N-fp. 



Since ^ G [0, 1] and > 0, 



i2 N^oo 



il + a')-'N-fp. 



(35) 



(36) 



Therefore, using /3i in place of /3o yields very little penalty at higher blocklengths and satisfies the power constraint 
as 00. 



B. Optimization Over Power Constraints 

Taking another look, the linear coding scheme described in the previous section can be further optimized if now 
we assume that 7 is not fixed. This will give us another degree of freedom in attempting to maximize the received 
SNR. Unfortunately, as stated above, a closed form expression for f3o is unavailable, so we solve for the solution 
for power allocation using /3i. 

Lemma 6. The power allocation scheme that maximizes received SNR, using jSi, can be found using the following 
method: 
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1) Define: 

• a = (7^, 

. b = p{l + a^). 

2) Let the optimal 7 G [0, 1], 70, be the smallest positive root of 

a(l + 57)^-iV6(l-7) + (6+l), (37) 
if it exists. If not (when ( 1471 ) is true), 70 = 0. 

Proof: From above, the received SNR for our scheme is of the form 

^NE(B + (l + a^)A^(l-7)p 

SNR[fi^,-i) - (38) 

Ignoring the constants in the numerator and using the definitions in the lemma, maximizing (l38l l over 7 is equivalent 
to 



1-7 



a+(l + 57)-(^-i) 
After taking the derivative and setting to zero, we get 



(39) 



a(l + &7)''' -7V6(l-7) + (fo+l) = 0. (40) 
Note that is possible to get no root that lies in [0, 1]. This occurs when 

p(l + cr^) 

In this case, the value of 7 reflects that noise-cancellation is no longer useful, and we set 7 to zero. ■ 
A graph showing the behavior of 70 versus p can be seen in Fig. |4] and a plot of 70 is given in Fig. |5] Note that 
the label linear units is used to emphasize that the axis is plotted on a linear scale and not in dB. The plots show 
the behavior of 70 with varying levels of feedback noise. In both increasing either p or A^, it can be seen that 70 
decays to zero eventually. As N increases, the additional use of feedback introduces more noise into the system, 
so at higher feedback levels the 70 will not peak as high and decay more quickly. As p increases, the numerator 
of the received SNR begins to predominate the maximization and 7 decreases to maximize (1 — 7) accordingly. 

An important sidenote is to mention the behavior of this scheme (with optimal 7 and (3) in the absence of 
feedback noise. It turns out that as — > 0, the method above produces the form of the solution derived in |5| as 
the optimal linear feedback scheme for the AWGN channel with noiseless feedback. However, in the presence of 
feedback noise, this solution noticeably differs. 

C. Further Analyses of the Linear Feedback Scheme 

In this section, we examine our scheme under different circumstances to derive results in related papers. 
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Fig. 4. A look at the behavior of 70 versus power constraint p. 




Fig. 5. A look at the behavior of 70 versus power constraint A'^. 
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1) Asymptotic Performance: Using we can examine the asymptotic behavior of our scheme as — > oo. If 
we let 7 — -iff, then the received SNR can be written as 



SNR(P,,^) = ^^^J^^^^ "^1^1 , (43) 



1 



N^oo 1 + cr^ 



(44) 



Np. (45) 



The received SNR of our scheme meets the upper bound in (1251 1 as — > cx); therefore, our scheme is asymptotically 
optimal. It is worthwhile to note the choice of 7. For this bound to appear asymptotically, 7 needs to be chosen as 
a function of N such that Nj 00 and 7 — > as 00. Note that these constraints were motivated empirically 
by the behavior of 70 which is found numerically. If 7 is not chosen within these constraints, this bound does not 
apply. 

2) Binary Communications: Now consider using our scheme to transmit a binary (M = 2) symbol, 6. The 
probability of error in such a scheme can be shown to be 



Pe^Q (^VSNRj , (46) 



which as — > 00 is 

This expression can be bounded above by 

Q 



Pe^Q\ \i^-^Np] . (47) 







j <^exp 





(48) 



By definition, the error exponent for a given is 



^(binary, p, a^) = lim -lln(Pe), (49) 



which in our case is 

'1 



i;(binary, p, cr ) = lim -^In 7;'^^P 



(50) 



This exponent simplifies to 

2a 



^(binary, p,a^) = ^ (51) 



This result meets the upper bound of the error exponent found in ifTSl and therefore shows that our scheme 
asymptotically achieves the highest rate of decay of probability of error. An illustration of this can be seen in Fig. 
|6] This simulation was run with with exact values of /3o and 70 which were found numerically. 

In ifTSl . a three-phase scheme is proposed that achieves this error exponent. In brief, the message is transmitted 
in the first phase and, using feedback, the transmitter decides whether the receiver made the right decision. The 
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Fig. 6. Comparison of the probability of error for binary transmission of the new scheme and the error exponent upper bound given in [18| . 

transmitter will then send one bit to the receiver stating whether the first transmission was a success or a failure. 
If the transmitter decides the receiver made a wrong decision, it declares a failure and retransmits a high-power 
version of the original message; otherwise, it declares a success and does nothing. It is important to note that this 
scheme was proposed in a general setting and was not constricted to binary transmissions. 

V. Simulations for the Inner Code 

We now present simulations to demonstrate the performance gains from our scheme and also the effects of 
feedback noise. The following analyses were verified using Monte Carlo simulations. 

A. Linear Feedback Comparisons 

In this section, the performance of the proposed linear scheme is compared with the Schalkwijk-Kailath (S-K) 
scheme (as discussed in Section I) under different circumstances. The first simulation (Fig. |7]i plots the received 
SNRs for both our scheme and the S-K scheme versus the transmit SNR, p. The value of optimal /?, /3o, was found 
numerically and used to construct our scheme. The feedback channel noise has variance a'^ — 0.01 and the power 
allocation is not optimized. Since power allocation was not optimized, both schemes are using 7 = (the value 
as given in the S-K scheme). As can be seen, with these assumptions, our scheme shows an approximately 2 dB 
gain over the S-K scheme in the low p regions {p w 1). Note that the p axis is not in dB but a linear scale to help 
show the difference in performance. 

The next simulation (Fig. ^ compares again the received SNR of the two schemes but for higher feedback noise 
{(7^ = 3) without power optimization (7 = ■^^^). This shows quite a difference from the low feedback noise 
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Fig. 7. Comparison of the new scheme and S-K scheme with low feedback noise (without power optimization). 



case. Both schemes suffer a drop in performance, yet the separation between the two schemes is larger. Another 
difference worth noting is the saturation of both schemes based on blocklength. At higher feedback noise levels, 
blocklength does not greatly affect the performance as can be seen by the grouping of both sets of curves. In fact, 
this phenomenon is due to the fact that we are using 7 = If we look at the received SNR{l3i, ■^^^) for our 

scheme as ^ 00, we can see that 

SNn(,,.^) . "".'^'^"-n",. . (52, 



N 



N-1 



N-1 ^ 



-{N-1) ' 



(53) 



N^cx. 



a2 



-P- (54) 



This is a tight bound for the received SNR when using the S-K power allocation with our scheme. 

The next figure displays the effects of optimization of power allocation. We see from Fig. |9] that power allocation 
has greatly increased the performance of our scheme compared to the S-K scheme (still fixed at 7 = n^)- This 
performance increase also appears to depend on blocklength. At A^ = 3, our scheme shows improvements in the 
range of 2-4 dB, but when A^ = 10, we see improvements in the range of 10 dB. This is because it is no longer 
constrained by (|54| |. Because of the new choice of 7, it can now reach the (^ ^^2 ^ Np bound. 

The last figure. Fig. [TOl shows how the received SNR of both scheme behaves with increasing feedback noise. 
As is evident in the figure, the proposed linear feedback scheme is much more resilient to the effect of growing 
feedback noise. Power allocation was optimized in this simulation and the average transmit power is p = 1. 
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Fig. 10. The proposed linear feedback scheme displays resilience in the presence of increasing feedback noise. 



VI. The Concatenated Coding Scheme 

Now that an appropriate inner code has been designed, it is possible to evaluate the performance of the total 
concatenated code. For the following derivations, it is still assumed that the outer code is a general error-correction 
code. In the next two sections, the error exponent for the concatenated code scheme is studied as the error exponent 
is an important measure of performance. Upper and lower bounds for the error exponent are derived to illustrate 
the advantages of implementing feedback. 

A. Feedback Error Exponent Lower Bound 

The goal of this section is to find a lower bound on the reliability function for the closed-loop concatenated 
scheme. To do this, we consider the best possible use of the proposed linear feedback scheme as an inner code. To 
begin, let our choices for (3 and 7 both be optimal such that /3 ~ fio from Lemma 5 and 7 = 70 from Lemma 6 
(i.e., ElO'^O] = KN{1 — ja)p). As discussed in Section II, the problem can now be transformed into designing a 
K channel use code for a non-feedback AWGN channel with SNR 

bNR{N,cr ,p) - 2(N-i) ' ^55) 

where the SNR is now only a function of N, cP' , and p (implicitly both 70 and /3o are also functions of N , a^, 
and p). 

Utilizing this non-feedback channel, we will now derive the error exponent expression using the open-loop 
reliability function. The open-loop reliability function is defined as the rate of decay of probability of error for the 
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best possible length K coding sequence across a non-feedback channel or 

EnoFB {R;P)-- lim sup - ^ In (i?; P) , (56) 

coding at a rate of R (bits/channel use) with a received signal-to-noise ratio P and achieving a probability of error 
of Pe{R\ P)- Now, implementing the optimal open-loop code as the outer code over the new non-feedback channel, 
we achieve an open-loop error exponent of 

^Ei,,FB {NR; SNR{N, a^, p)) . (57) 

The rate scaling by N is due to the fact that our total blocklength has increased by a factor of N, but at the 
same time, we can only send a new symbol every N channel uses. Also, because of this structure, a trade-off in 
error exponent performance arises as one varies the value of N. SNR{N, ct^, p) grows with increasing N which is 
favorable, but, simultaneously, the rate increases and the factor of decreases with increasing N - both adversely 
affecting the error exponent. Because of this trade-off we will now define the optimal N, N* that achieves the 
highest value of the error exponent. 

N* = argsup ^EnoFB {NR; SNR{N, a^, p)) . (58) 

N=l,2,... 

We can now, using (|57] |. define the closed-loop concatenated code error exponent, EpB, by 

Efb {R; P, fr') = -^EnoFB {N*R; SNR{N*,a^p)) . (59) 

With this result, we can now examine the concrete bounds on the error exponent for the concatenated scheme and 
thus create bounds for feedback error exponent. For all rates below capacity, we can employ the random coding 
lower bounds ||20l on the error exponent. 

Lemma 7. If we first define: 




Then, we can write the concatenated code error exponent lower bound as 

EMR,P,<^')>\ f ^I'-'^Z'C-i (1 - VI - + ^-R, R^<R<R2 

I (T'^+p -"-1 

[ j^Es,{N*R, (li^jpiz^), R,<R<C 
where Egp(R,P) is the sphere packing bound given in [20], N* is chosen at each value of R to maximize the 
bound, and N'^_^ is the value of N* chosen at Ri. The optimal inner code blocklength, N* should scale as 

N*^0 . (60) 
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In addition, N*, for low rates, can be approximated by 



N* « 

where 9^ = arcsin (— A^i?). 



root 



2(Ar3/2 _ C0s(6>jv)(l -C0s(6lAr)) 



,0<R<Ri (61) 



Proof: The bounds given in the lemma are a direct application of the random coding lower bounds in 
The approximation for N* is derived as follows. To avoid exceeding capacity, the constraint N*R < C must be 
imposed. With this constraint, we can now build an approximation by looking at the expression for low rates (i.e., 
< i? < The SNR expression imposed by the use of linear scheme is quite difficult to maximize over N due 
to the reliance on (3 and 7; therefore, to approximate it, we can replace it an approximation that only relies on 7 
and set 7 — as earlier. Then, we can write 

Kppro. = argmax ^ (1 - VT^T^) , (62) 

and find the "optimal" N (by differentiating and setting the derivative to zero) is given by the root of 

, 0/9 , cos(0/v)(l — cos(6'fv)) 
2{N^/^-N)^ ^ ; ^" , (63) 

Rsm [Off) 

where 9n = aicsm{—NR). ■ 

Lemma 7 gives explicitly the random coding lower bounds for the concatenated coding scheme. For completeness, 
the sphere-packing bound will now also be defined. If we first let 9{R) — arcsine^^, then the sphere packing 
bound can be given concisely as 

g{9,P) = i (yPcos{9) + ^Pcos2(0) +4 

The error exponent lower bounds as given in Lemma 7 can be seen in Fig. [TT] Note that the label "no feedback" 
refers to the error exponent of purely the outer code with no inner code. 

The approximation (without the floor operation) can be seen versus the numerically optimized N* in Fig. [12] 
This gives us a rough handle on how feedback should be used (in the asymptotic sense) for the concatenated coding 
setup. Namely, it should be used only at low rates, but can dramatically increase the error exponent bound at these 
rates as seen in Fig. [TT] 

B. Feedback Error Exponent Upper Bound and Special Cases 

Just as important as investigating the effect of feedback on error exponent lower bounds is the effect on the 
upper bounds. In this case, we employ the use of two well-known error exponent upper bounds, the minimum- 
distance upper bound, E,nd{R,P) El], li22l . in conjunction with the sphere-packing bound 120|. Note that the 
sphere-packing bound gives the exact expression for the error exponent in the R2 < R < C region. For reference, 
the minimum-distance bound can be given as follows: 

E{R, P) < E^4R, P) = ^(P{R), (64) 

o 
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Fig. 1 1 . En'or exponent bounds for non-feedback schemes and the proposed concatenated coding system. 




Fig. 12. The TV* approximation compared to the actual optimal values. 



DRAFT 



23 



where cP{R) is the squared minimum distance of the code at rate R. This can be given an upper bound as in ll22ll 
by first defining 5* as the root of i? = (1 + S)H{ j^) and H{x) = -a;lnx - (1 - x) In (1 - x). Then, 

m < (65) 

To ensure tightness, we take the minimum of both bounds at any given rate, R. Hence, the following function 
which was used to plot the upper bounds on error exponents in Fig. [TTl 

EFBiR,P,cT )< mm ^—E^d{N R, _^ ), — ^,p(iV R, _^ ^^(jv.-i) ) j > < i? < C. 

Again, the value of N* is chosen to maximize the bound at any given R. As in Fig.[TTl he upper bound for feedback 
is higher than the upper bound in the absence of feedback. This gap closes as feedback noise variance increases. 
As noted earlier, when N > [^J or when R > ^, feedback should not be employed as it exceeds capacity - this 
is noted in the graph. 

The main idea introduced by this section (and the previous), is that the implementation of feedback can allow 
for a new tradeoff - explicitly between rate and received SNR - that can exploited for further increases. Of course, 
this tradeoff becomes less useful as the feedback noise increases, but it still creates a new degree of freedom. Also, 
another conclusion one can derive is that feedback is very beneficial at low rates. This will be substantiated further 
by simulations in Section WU\ 

Now, the error exponent of the concatenated coding scheme is given in the special cases of i? = and where 
feedback is no longer useful. 

Lemma 8. Special Cases 

1) At R — 0, the error exponent for the above concatenated scheme is: 

EFBiR = 0,P) = ^^^^p. 

2) Also R>\ log2 (l + 2(i+<T^Ki-7o)p j^ j/^g g^^^^ exponent is 

Efb{R, P) — EnoFb{R, P), 

i.e., feedback is not used. 

Proof: At rates very close to zero, we can solve analytically for the error exponent of our scheme. It is a 
classic result that for R — 0, the following is true 



EnoFb{R^O;P)^^. (66) 



This would imply i5% can be written as 



EFB{R = 0;P,a^) > l^SNR{N,a^,p), (67) 

1 (l + a2)7V(l-7o)p ^^g^ 
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Note, however, that as i? — > 0, we can let — > oo. As stated earlier this implies 7 — > and — > since 
/3 e (0,1). This produces 

EpB {R = 0;P,a^) - > ^ = EnoFbIR - 0,P). (69) 

For the second result, let us look at the specific case of iV = 2. Fortunately, when iV = 2, we can solve 
analytically for /3 using Lemma 2. After some algebra, it is 



Using this value of (3, the received SNR is calculated to be 



snrwo.,) = (71) 



2 ' "'^ V 2 



(72) 



For a rate to be achievable, it must satisfy 

NR < log2 (1 + SNR{l3o, 70)) , (74) 

where 70 is the optimal 7 defined in Lemma 6. Setting N — 2 and using ( l73T l. feedback should not be employed 
with our concatenated scheme if 



VII. Simulations for Concatenated Coding 

In this section, the performance of the concatenated coding system in Section VI is simulated for the cases where 
the outer code is an LDPC code (according to WiMAX standard) and a turbo code (according to UMTS standard). 
Details for each code are given below. These simulations were run using the Coded Modulation Library ll23l . To 
keep the number of channel uses consistent, the concatenated coding scheme has to implement 2^ modulation 
order versus the open-loop technique using BPSK. Therefore, to use an inner code of 2 iterations of the proposed 
linear scheme, 2^ modulation order is used (i.e., QPSK). Also, to ensure that both schemes use the same average 
power, the linear feedback scheme must be designed with a particular value of p. In particular, if the open-loop 
technique uses Eg energy per symbol to modulate, then the linear feedback code must be designed with p — Ef, 
when using 2^ modulation order Fig. 10 shows that the ^ required to attain a certain probability of bit error is 
up to 0.4 — 0.5 dB lower by using the concatenated coding scheme for the UMTS turbo code (Rate 1/3, K ~ 5114 
bits, 10 decoding iterations). Note that this turbo code uses the max-log-MAP algorithm. This gain in performance 
is also a function of the feedback noise variance. As can be seen, the performance gains diminish as feedback noise 
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Fig. 13. BER performance of concatenated coding scheme versus open-loop coding for UMTS turbo code {K = 5114 bits, Rate = 1/3, and 
10 decoding iterations). 
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Fig. 14. PER performance of concatenated coding scheme versus open-loop coding for UMTS turbo code (K = 5114 bits, Rate = 1/3, and 
10 decoding iterations). 
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Fig. 15. BER performance of concatenated coding scheme versus open-loop coding for WiMAX LDPC code {K = 2304 bits, Rate = 5/6, 
and 100 decoding iterations). 



increases. The same phenomenon is apparent in Fig. 1 1 which is a comparison of the frame-error-rate (FER) for 
both techniques. A similar improvement (up to 0.4 — 0.5 dB) can be seen. 

Fig. 12 display the the BER for both the open-loop and concatenated coding scheme using the LDPC code as 
given in the WiMAX standard (Rate 5/6, K = 2304 bits, 100 decoding iterations). Again, the concatenated code 
is modulated using QPSK and has an inner code of two iterations of the proposed linear feedback scheme. The 
performance of the concatenated coding scheme again is much better than pure open-loop techniques - displaying 
up to around 2 dB improvement. The effect of feedback noise is clear as it greatly closes the gap between the two 
methods. However, it is interesting to see that the N — 3,a^ — 0.001 performs much better when compared to the 
turbo code. Fig. 13 displays the FER for both schemes which demonstrates up to around 2 dB improvement. 

An interesting point introduced by extending an open-loop error-correcting code into a closed-loop concatenated 
code is the tradeoff between modulation order and increases in received SNR for the channel. As one increases 
the number of iterations of feedback coding, the received SNR increases. However, simultaneously, the modulation 
order increases which creates a less forgiving probability of symbol error This tradeoff allows for a new degree of 
freedom in transmission schemes that can be exploited to achieve lower error rates. 

VIII. Conclusions 

In this paper, we investigated a specific case of concatenated coding for the AWGN channel with noisy feedback. 
The inner code was designed as a linear feedback scheme that was constructed to maximize received signal-to-noise 



DRAFT 



27 



10 1 ^ m 



,,,:+: 



— o. . .P 







- * - QPSK, N = 2, a"^ = 2 

- + - QPSK, N = 2, tj^ = 0.5 

- V - 8PSK, N = 3, = 0.001 

- e - BPSK, No Fdbk 



10" 



10" - 



10 ' 



I 

V 



1 1.5 



2.5 



3.5 



E,/No(dB) 



Fig. 16. FER performance of concatenated coding scheme versus open-loop coding for WiMAX LDPC code {K = 2304 bits, Rate = 5/6, 
and 100 decoding iterations). 



ratio. The performance of the linear feedback scheme was compared to another well-known feedback technique, 
the Schalkwijk-Kailath scheme. The outer code was allowed to be any open-loop error correction code for ease of 
adaptation. The concatenated coding scheme shows that the use of feedback can greatly increase error exponent 
bounds compared to pure open-loop techniques. Simulations illustrated the performance of the linear scheme and 
its incorporation into the concatenated coding scheme when the outer code is either a turbo code or LDPC code. 

Appendix 

A. Schalkwijk-Kailath Coding Scheme 

The S-K scheme is a special case of the linear feedback encoding framework formulated in Section III. When 
describing the S-K scheme we will ignore feedback noise {<t^ 0), since it was designed for a noiseless feedback 
channel. In the S-K set-up, 7 — -^^^ and g, F, and q have the following definitions: 

1) g= [1,0,...,0]^, 
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2) Let = 1 + p and r = Then F is an x iV encoding matrix given by 











3) 



q = 



9 ' T' • ■ • ' AT 



B. Optimization of Received SNR 

For this optimization, let us assume that 7 is fixed and without loss of generality, g and q are both unit vectors. 
With those assumptions, the goal at this point is to design g, q, and F to maximize ( fTSb . Looking first at the 
numerator, we see that we can bound |q^g|^ using the Cauchy-Schwarz inequality. Doing this, we see that 

2 II ii2 



I T |2 

q g 



< llq|l 
= 1. 



This bound can be achieved by letting g = q. For our purposes now, we will always assume that g — q, F is 
restricted as in ( fT3l ). and E[9'^] = N{1 — ^)p. With these conditions, the received SNR were are trying to optimize 
simplifies to 



SNR- 



N{l-l)p 



|q^(I + F)f + (T2||q^F||'' 



(76) 



Note also that in the S-K case, even though q is not a unit vector, still [q^'^gl = 1. 

Since the numerator is now fixed, our focus now turns towards minimizing the denominator However, this is 
more complicated. The ideal solution would be to jointly minimize the denominator over q and F. Unfortunately, 
this does not yield any feasible path towards a solution. Instead of attempting to jointly optimize, we now derive 
the two conditional optimization methods used as Lemma 1 and Lemma 2. 

First, consider minimizing the denominator given a combining vector q. Since q is given, the goal is to design 
F to maximize ( fTSl ): therefore we should pick F using 



' opt 



argmm 
F 



q^ (I 



subject to ||F||^ < (1 + (r'^y^N^p and ^ when i < j 



(77) 



We now have sufficient background to prove Lemma 1 . 
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Proof: (Lemma 1) To begin let us define the non-zero columns of F as = [ fi+i^i, fi+2,i, ■ ■ ■ ,fN,i]^ for 
1 < i < iV — 1. Now, working through the multiplication, we can rewrite 



N-l 



|q^(I + F)f =^(9. + q^)fO^ 



(78) 



At this point, it is worthwhile to remark that minimizing this sum is equivalent to minimizing the total sum given 
in ( ITTI i. This is due to the fact that the subspace for the solution of F in the first term is that same as in the second 
term, cr^||q-^F||^. This can be seen as both terms can be written in the form ||Fq + b||^ where in the first term 
b = q and in the second b = 0. Both solutions can be carried out the same way with the assumption that b > 0, 
which is assumed. For lack of redundancy, only the minimization of the first term is explicitly carried out. 



Looking back, to minimize (iTSl l. we need to minimize qi + qj'-^ii for all i. This can be accomplished by designing 



the {f;} such that 



(79) 



where 



N-l 



(80) 



The introduction of {ai} is required because of the constraint, ||F||^ < (1 + o" ) Njp. Substituting in for the 
new columns of F produces 

||q^(I + F)f - ^fe- ||q(,)||aO' + gw. (81) 

This limits the problem of designing the matrix F to finding the {ai} that minimize dSTl ) and satisfy dSOl ) - this 
is a norm-constrained least squares problem. This is more evident if we let 

||q(i)|| ••• 

||q(2)|| ••• : 

■■■ 

||q(iv-i)|| 















and b = [ai,a2, ■ ■ ■ ,cxn-i]^ ■ Thus, rewriting dSTT l. the problem of minimizing the ||q-^(I + F)|| term now 
becomes 

min l|Ab — qj]^ . 

subject to ||b||^ < (1 + a^^^N-fp 
Noting that q^(I + F) = (q — Ab)^ and q^F = (— Ab)-^, we can calculate the optimal b using 



Ab-q|p + cr^||Ab|r. 
subject to ||b||^ < (1 + (r^y^N^p 



Jopt = argmm 

b 



(82) 
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At this point, the focus is now not only on the first term but taken over the whole sum in ( |77] l as can be seen with 
the introduction of the <t^ term in (l82T i. To solve for the optimal b and make sure that ||b|r < {l + (7^)-'^N-/p, 
we use Lagrange multipliers. Forming the Lagrangian, we get 

L{h, A) = q^q - 2b^A^q + b^A^Ab + a^b^A^Ab + A(b^b - (1 + a^)~^N-fp). 
After taking the gradient with respect to b and setting to zero, solving for the optimal b results in 



bopt = ((1 + <7^)A^A + AI)-^A'^ q. 



-1 aT, 



(83) 



where A is chosen such that b^b = (1 + cr^) ^N^p. Once b has been calculated, F can be constructed using ( 1791 ). 



To prove Lemma 2, we consider the case when F is given and we are designing q to maximize the received 

SNR. The goal now is to find q such that 

qopt = argmin ||q^ (I + F) ||^ + cr^ ||qTF||2 
q 

subject to ||q||^ = 1 

This problem, however, can be solved very quickly as given in the following proof of Lemma 2. 

Proof: (Lemma 2) Let 52, . . . , 5n be the eigenvalues of (I + F)(I + F)^ + cr^FF^ such that 5i> 52> 
...>5n>Q- Then, 

||qT(i + F)||2 + ^2 ||qTp||2 ^ ^(j + F)(i + f)T ^ ^2ppT] q 

>&N- 



This bound can be achieved by letting q be the eigenvector of (I + F)(I + F) + cr FF corresponding to 5 



N- 



This choice of q leads to q-* (I + F) + a q F — 5_ 



N- 



These two conditional solutions allow for numerical optimization as discussed in Section IV. 



C. Proof of Lemma 5 

We now provide the proof for the structure of our linear scheme as given in Lemma 5. 

Proof: To find the entries of F, let us consider entries /jv_i jv_2 and fN,N-i shown below; 





/2,1 



N.l 







/Ar-l,Ar-2 
fN,N-2 /jV,7V-l 



From the form in Conjecture 1, we should have that 

fN-l,N-2 = Jn.N-I- 



(84) 
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Now we use Lemma 1 to begin finding the form of F given the exponential form of q. Using step 3 of Lemma 1, 
we compute b as 



A+(l+<T2)||q(l)|| 
A + (l+<T2)||q(2)|| 



(85) 



A+(l+(T2)||q(„_i)f 

Now, using the definitions of the columns from step 4 of Lemma 1, we get 



^-1,^-2 — 



TN,N-1 



X+ (1 + Cr2) ||q(Ar_2)| 
A+(1 + Cr2)||q(^,_i)| 



Then, using ( l84l ). we solve for A which produces 



A 



{l + (7^){(3^\\q^N-2)\\ -||q(w-i)||) 



1 



(86) 
(87) 

(88) 



Since the form of q consists of consecutive powers of /?, we can state the following; 

N-l N-1 

||q(Ar-2)f -||q(A^-i)f = E E 

i=N-2 i=N-l 
^ ^2(7V-2)^ 



(89) 



Using the value of A from dSSl l in b and simplifying using ( |89] l results in the [N — 2)*'' component of b being 

lq(iV-2)||/3^-^(l-/3') 



&JV-2 



(l+^2)^2(N-2) 



Using 6jv-2 to construct fAr-2, we find 



lAr-2 



fN-l.N-2 
fN,N-2 



|q(JV-2) 



-1 



(l + a2);32(JV-2) 

1-/3 



|q(Af-2) I 



pN-2 



(1+^ 

Using this pattern we find that any non-zero column of F can be written as 

(l+a2)/3 
(1+^ 



(90) 



fi+lA 




fi+2,i 




In, I 





which completely defines the structure of F. 
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Utilizing this structure of F, the Frobenius norm of F can be computed to be 



(l + a2)2 



(91) 



Using this result and the bound |1F||^ < (1 + cr^) ^N^p, we find that the 13 that meets the bound is the smallest 
positive root of 



- (TV + (1 + a')N^p)p' + (iV - 1). 



(92) 
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